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(54) Method of updating reference background image, method of detecting entering objects and 
system for detecting entering objects using the methods 



(57) A method of updating a reference background 
image used for detecting objects entering an image 
pickup view field based on the binary image generated 
from the difference between an input image and the ref- 
erence background image of the input image. The 
image pickup view field is divided into a plurality of view 
field areas (201), and a reference background image 
corresponding to each of the fixed divided view field 
areas is updated (204). An entering object detection 
apparatus using this method has an input image 
processing unit including an image memory (603) stor- 
ing an input image from an image input unit, a program 
memory (606) storing the program for activating the 
entering object detecting unit, a work memory (604) and 
a central processing unit (605) activating the entering 
object detecting unit in accordance with the program. 
The processing unit has an entering object detecting 
unit (104) determining intensity difference for each pixel 
between the input image and the reference background 
image not including an entering object to be detected 
and detecting an area with the difference larger than a 
predetermined threshold as an entering object, a divid- 
ing unit (201) dividing the image pickup view field of the 
image input unit into a plurality of view field areas, an 
image change detecting unit (202) detecting the change 
of the image in each of the divided view field areas, and 
a reference background image updating unit (204) 
updating each portion of the reference background 



image corresponding to each of the divided view field 
areas associated with the portion of the input image free 
of the image change, wherein the entering object 
detecting unit detects entering objects based on the 
updated reference background image. 



FIG. 1 
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Description 

BACKGROUND OF THE INVENTION 



[0001] The present invention relates to a monitorina svstem nr mnw in r>^r*;~ i * 

method and an entering object detecting sy^T^^^^Z SEffi?* 8 

E£S T eW " V6hiC,eS mWin9 in ,he ima9e picku P view ,ieW f ™ ^ge Snal ""^ 
10002] An image monitoring system using an image pickup device such a* a rarJL ^ ~ * .. t 

used. In recent years, however, demand has arisen for an cWacSn^ bee " ^ 

itoring system by which objects such as persons or autSilStSTZ^ - ™ ™°' 
detected from an input image signa. and prLtermined IZSToSSS i^ZS^ ^ 
son viewing the image displayed on a monitor produced automatically without any per- 

K^pde^eT^ £ *- Gained .cm „ 

be detected thereby to detect a difference in intensity 22? S'" 9 ^f? 9 **** to 

sKydtfe rence is detected as an entering object. SEn-A ^slSon^ S ^7 

change in the monitoring view field, for e£mS? £^32^222^^ ^ *" * i,luminance 

ance with the iUuminance change background -mage ,s required to be updated in accord- 



SUMMARY OF THE INVENTION 



[0005] Several methods are available for updating a reference background image Thev incluri* a mo *^ ^ ^ 
ing a reference background using an average value of the intend w oa T" ,t7- ^ C "* a metnod for P roduc - 
(called the averaging method), a method for s ^ 

weighted average of the present input image a^T^^Z^L ^J^ background image from the 
termined weight (called the add-i* Method) a ^t^^SlT^ ' ma9e ' calculated und * a P'ede- 

the intense of aU pixel nTvirfg ^7? °* 

this process is executed for all the pixels in a monitoring area ^1 to ^^JSSW 1^* ^ and 
reference background image is updated for pixels other than tn to^7erte^h^?^ J T*** ,n •»** ^ 
traction method (called the dynamic area updating method) * ^ and det6C,ed * 8,6 sub - 

[0006] In the averaging method, the add-iiD method and thp morion u 

producing a reference background image £d a torn tirne ten ^ThTf T""* frames are rec * uired fw 
ground image after an input image SeTanv T^lnl ^ C ° mp,ete Upda,i " 9 of the reterenca back- 

forancb^tracng aTZ^ KjSJ ^X^n^^S^ fe ^ 
match occurs in the boundary between Dixels with the rrf^JT^Z V? ' 6 olher hand ' a inten sty mis- 
ence background image not updated to o^^^^^^^^ •» -fer- 

falsely looks as if a contour exists at a portion where ^ tm£oun^^Z t^^^T™™ *" * 
due to generation of a stepwise intensity change at an interface bX^ I ^ ,ntenSrty 
specifying the position where to mismatch has occurred the pastTmaaS o?SSS2 ^ n0t upda,ed - For 
^stored, so tot an in.ge s^age m_y „ a terge ^ ^T^^ZZ^X 

j^me^ 

[0008] Another object of the invention Is to provide a method an H ^ cur*- ~ . . , 

image in accordance with to brightness or £££ in^ ^^S^TST* 

of a small capacity. ^ ue; uiange or an input image using an image memory 

[0009] Still another object of to invention is to provide a method and a s «t M fc,,^ «. . 

which an intensity mismatch which may occur beKveen to S^uodat^^^ 22* ^ tl * back 9 round ima ^ * 

'ssnzr I s to ^ on ,he re,iatt,Hy for «^^ssi^^ updated * *• re,erence 

Son i£S ^ ° f ^ ^ fe to ^ a ^od and a system for detecting entering cbjec* high in 

and the portfon of to reference background corresponding to JSa^SSS - "> 3 ^ 

10121 7116 ,ma9e P,CkuP V,6W ,ieW ^ be divided a ^ ^ reference baokgroundlge for each dhrWed area may 
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be updated after detecting an entering object Alternatively, after dividing the image pickup view field, an entering object 
may be detected for each divided view field and the corresponding portion of the reference background image may be 
updated. 

[001 3] Each portion of the reference background image is updated in the case where no change indicating an entering 
5 object exists in the corresponding input image from an image pickup device. 

[0014] Preferably, the image pickup view field is divided by one or a plurality of boundary lines substantially parallel 
to the direction of movement of an entering object. 

[001 5] Preferably, the image pickup view field is divided by an average movement range of an entering object during 
each predetermined unit time. 

w [001 6] Preferably, the image pickup view field is divided by one or a plurality of boundary lines substantially parallel 
to the direction of movement of an entering object and the divided view field is subdivided by an average movement 
range of an entering object during each predetermined unit time. 

[001 7) According to an embodiment, the entering object includes an automobile, the input image includes a vehicle 
lane, and preferably, the image pickup view field is divided by one or a plurality of lane boundaries. 
75 [0018] According to another embodiment the entering object is an automobile, the input image includes a lane, and 
preferably, the image pickup view field is divided by an average movement range of the automobile during each prede- 
termined unit time. 

[001 9] According to still another embodiment the entering object is an automobile, the input image includes a lane, 
and preferably the image pickup view field is divided by one or a plurality of lane boundaries, and the divided image 

20 pickup view field is subdivided by an average movement range of the automobile during each predetermined unit time. 
[0020] According to a further embodiment the reference background image can be updated within a shorter time 
using the update rate of 1/4, for example, than by the add-up method generally using the lower update rate of 1/64. 
[0021 ] According to another aspect of the invention, there is provided a reference background image updating system 
used for detection of entering objects in the image pickup view field based on a binarized image generated from the dif- 

25 ference between an input image and and the reference background image of the input image, comprising a dividing unit 
for divicfing the image pickup view field into a plurality of view areas and an update unit for updating the reference back- 
ground image corresponding to each of the divided view fields independently for each of the divided view fields. 
[0022] According to still another aspect of the invention, there is provided an entering object detecting system com- 
prising an image input unit, a processing unit for processing the input image including an image memory for storing an 

30 input image from the image input unit, a program memory for storing the program for the operating the entering object 
detecting system and a central processing unit for activating the entering object detecting system in accordance with 
the program, wherein the processing unit includes an entering object detecting unit for determining the intensity differ- 
ence for each pixel between the input image from the image input unit and the reference background image not includ- 
ing the entering object to be detected and detecting the binarized image generated from the difference value, i.e. 

35 detecting the area where the difference value is larger than a predetermined threshold as an entering object, a dividing 
unit for dividing the image pickup view field of the image input unit into a plurality of view field areas, an image change 
detecting unit for detecting the image change in each divided view field area, and a reference background image update 
unit for updating each portion of the reference background image corresponding to the divided view field area associ- 
ated with to the portion of the input image having no image change, wherein the entering object detecting unit detects 

40 an entering object based on the updated reference background image. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0023] 

45 

Fig. 1 is a flowchart for explaining the process of updating a reference background image and executing the proc- 
ess for detecting an entering object according to an embodiment of the invention. 

Fig. 2 is a flowchart for explaining the process of updating a reference background image and executing the proc- 
ess for detecting an entering object according to another embodiment of the invention. 

so Figs. 3A, 3B are diagrams useful for explaining an example of dividing the view field according to the invention. 
Figs. 4A, 4B are diagrams useful for explaining an example of dividing the view field according to the invention. 
Fig. 5 is a diagram for explaining an example of an image change detecting method. 
Fig. 6 is a block diagram showing a hardware configuration according to an embodiment of the invention. 
Fig. 7 is a diagram for explaining the principle of object detection by the subtraction method. 

55 Fig. 8 is a diagram for explaining the principle of updating a reference background image by the add-up method. 
Fig. 9 is a diagram for explaining the intensity change of a given pixel over N frames. 

Rg. 1 0 is a diagram for explaining the principle of updating the reference background image by the median method. 
Figs. 1 1 A to 1 1 C are diagrams useful for explaining the view field dividing method of Figs. 3A, 3B in detail. 
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Figs. 12A to 12C are diagrams useful for explaining the view field dividing method of Figs. 4A. 4B in detail. 
DESCRIPTION OF THE EMBODIMENTS 

[0024] First, the processing by the subtraction method will be explained with reference toFin 7 Rn 7 - A - 
for explaining the principle of object detection bv the subtrarti™ nLJrJu^ g - 7 ' F,g - 7 ,s a d,a gram 
an input image f. numeral 702 a referenc^ 701 designates 

rized image, numeral 705 an image of an objed t4^S m ^T^ ^ 704 3 bina " 

Fig. 17. the subtracter 721 produces the dSnceTmlge 703 SfSZ ™ ^J 4 **** h 
between the input image 701 and the reference background imaoe 7S ™SL£ ^ * each pixel 

Pixels of the difference image 703 less man a pJS^TSSKE £52 X^S^ * 
not less than the threshold as "255" (the brightness of each pixel Mo£Z^!^Z 2 ° f ** pixeb 

image 704. As a result the human object ^S^uZl^Tj^Zl^^ V * ^ b,narized 
image 704. The reference background image is a bactarZri Zn! S 38 30 ,ma9e 705 in the binarized 
In the case where the intensity PrtensfJ^ll^?^^^^ entering °** to be **** 

background image, it is necessary that the number of the frames to ha ' ° rder to ob,ain 30 accurate 

pie. 60 (corresponding to the period of 10 seanTsl^^ « ! 6ra9in9 ^ 56 **• "*» for exam- 

10 seconds) is unfavorably gSeS - be^SnTe ^^Se^lS^tS: 9 3 ^ ,a9 (ab0Ut 
are inputted and the time at which subtraction orJ^L^lT^Z ■ background image generation 

lem arises such that it becomes '^^rxo ^^Z^^T^^ ° UB *° tWs «"» ^ * 
usable as a current background ima^otS^^ k ^ r ° Und ,ma9e is accu "*e enough to be 
fie* suddenly changes wC£Z^ ^P*— * the ^ 

[0026] Next, the add-up method wil, be e^nSwS Sfer^ce to FteT^ Y 9 f ^ ° f the *"* 
of updating the reference background image usina ^e add ^Jlv? 3 d ' a9ram ,or explainin 9 a method 

background image, numeral 802 Bnt^ZTn^^TJ^^ « 801 desi ^ a reference 
rate, numerals 805. 806 posters numeral 807 an TZZ Ttl 1 background .mage, numeral 804 an update 
the add-up method the w^W^ a^oe^f the oSr£ ' .Z™** 821 iS 3 wei9hted *™W ^ator. In 
termined weight (update rate^m^^ ^e^lZ^^T^f ^ WHh 3 ^ 

ground image 803 sequentially. This p^ess is ex^eS^by^S * 



r mA*- y) = (1-R) x r, 0 + R x f ^(x. y) 



5 



pixel position. In the case where the badJn.,nH !!. {x> y) IS a c °°rc«nate indicating the 

image 802. for exarrpfe Poster 805 anew in the fnpu, 

as by the poster 806. When the update n£wE£Z2^££ 11*"* back9round ima 9e 803 such 
within a short time against the t^^^^T^^t^^T ™ * *° ^ 
to a large value, however, the image of an entering object m 7Sy Z^^!^T '** 804 * S6t 

new reference background image 803 in the ino.rt inSZ tk» ^ 'S Present in the mput ,mage. is absorbed into the 
set to a value (1/64. 1/32^ 1 for ZLTftT!^ " Upd3te ra,e 805 is r «M>ed to be empirically 

new ref^ba^rou^gf 80^ TSS^S^ 2 uX^is^SSt ^ ? — 
producing the reference background image by the averarim^Z? ' for example ' « is «»**lenl *> 

image of 64 frames for each pL In theSL^XuSSe ^ ^ °* 3n inDut 

frames is required from the time of occurrence of a ctonTi n ll ^ ' ,he P^^ss for 64 

reftected in the reference ba<*grou^ g Tm° S Se 1^ a S ^ f *" T* ^ iS ^ 

Styv^wThr^r^^ 

horizontal axis represents the tim* ,1 Z Z TJ -"T'T lM : naiural number > for a given pixel, in which the 



» 
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obtained in Fig. 9 are arranged in the order of magnitude along the time axis, in which the horizontal axis represents 
the number of frames and the vertical axis the intensity value, numeral 904 the intensity value data with the intensity 
value arranged in the ascending order of magnitude and numeral 905 the median value. 

[0028] In the median method, as shown in Fig. 9, the intensity data 903 is obtained from an input image for the same 
5 pixel of predetermined N frames. Then, as shown in Fig. 10, the intensity data 903 are arranged in the ascending order 
to produce the intensity data 904, so that the intensity value 905 for N/2 (median value) is defined as the intensity of a 
reference background pixel. This process is executed for all the pixels in the monitoring area. This method is expressed 
as 

10 r t0+1 (x,y) = med{f i (x,y)} (2) 

t 0 ^t<t 0 -N 

where r K)+1 is a new reference background image 905 used at time point to+1, a reference background image at 
15 time point to. f© an ""Put image at time point to, and med 0 the median calculation process. Also, (x, y) is the coordinate 

indicating the pixel position. Further, the number of frames required for the background image production is set to about 

not less than twice the number of frames in which an entering object of standard size to be detected passes one pixel. 

In the case where an entering object passes a pixel in ten frames, for example, N is set to 20. The intensity value, which 

is arranged in the ascending order of magnitude in the example of the median method described above, can alterna- 
te tively be arranged in the descending order. 

[0029] The median method has the advantage that the number of frames of the input image required for updating the 

reference background image can be reduced. 

[0030] Nevertheless, as many image memories as N frames are required, and the brightness values are required to 
be rearranged in the ascending or descending order for median calculations. Therefore, the calculation cost and the cal- 
25 culation time are increased. An example of an object detecting system using the median method described above is 
disclosed in JP-A-9-73541 (corresponding to U.S. Serial No. 08/64601 8 filed on May 7. 1996 and EP 96303303.3 filed 
on May 13, 1996). 

[0031] Finally, the dynamic area updating method will be explained. This method, in which the entering object area 
705 is detected by the subtraction method as shown in Fig. 7, and the reference background image 702 is updated by 
30 the add-up method for the pixels other than the detected entering object area 705, is expressed by equation (3) below. 

3(x,y)e {(x,y)|d to (x,y) = 0} (3) 

1 10+1 

(x,y) = (1-R')xr t0 + R'xf lo (x,y) 

35 

where d© is a detected entering object image 704 at time point to, and the intensity value of the pixels having the enter- 
ing object therein are set to 255 and the intensity values of other pixels are set to 0. Also, r^ indicates a new reference 
background image 803 used at time point to+1 , r© a reference background image 801 at time point to, f^ an input image 
802 at time point to, and R* an update rate 804. Further, (x, y) represents the coordinate indicating the position of a given 
40 pixel. 

[0032] In this dynamic area updating method, the update rate R' can be increased as compared with the update rate 
R for the add-up method described above. As compared with the add-up method, therefore, the time can be shortened 
from when the input image undergoes a change until when the change is updated in the reference background image. 
In this method, however, updated pixels coexist with pixels not updated in the reference background image, and there- 
45 fore in the case where the illuminance changes in the view field, the mismatch of the intensity value is caused. 

[0033] Assume, for exarrple, that the intensity value A of the pixel a changes to the intensity value A' and the intensity 
value B of an adjacent pixel b changes to the intensity value B\ The pixel a having no entering object is updated toward 
the intensity value A' following the particular change. With the pixel b having the entering object, however, the intensity 
value is not updated and remains at B. In the event that adjacent two pixels a and b have substantially the same inten- 
se sity value, therefore, the presence of a pixel updated and a pixel not updated as in the above-mentioned case causes 
the mismatch of the intensity value. 

[0034] This mismatch is developed in the boundary portion of the entering object area 705. Also, this mismatch 
remains unremoved until the complete updating of the reference background image after the entering object passes. 
Even after the passage of the entering object, therefore, the mismatch of the intensity value remains unremoved, 
55 thereby leading to the inaccuracy of detection of a new entering object For preventing this inconvenience j.e. for spec- 
ifying the point of mismatch to update the reference background image sequentially, it is necessary to hold as many 
detected entering object images as the frames required for updating the reference background image. 
[0035] An example of an object detecting system using the dynamic area updating method described above is dis- 
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closed in JP-A-1 1-127430 published on May 1 1 . 1 999 (Japanese Patent Application No. 9-291 91 0 filed on October 24 
1997). 

[0036] Now, embodiments of the present invention will be described with reference to the drawings. 
[0037] A configuration of an object tracking and monitoring system according to an embodiment will be explained Fig 
6 is a block diagram showing an example of a hardware configuration of an object tracking and monitoring system In 
Fig. 6. numeral 601 designates an image pickup device such as a television (TV) camera, numeral 602 an image input 
interface (l/F). numeral 609 a data bus, numeral 603 an image memory, numeral 604 a work memory, numeral 605 a 
CPU. numeral 606 a program memory, numeral 607 an output interface (l/F), numeral 608 an image output l/F, numeral 

610 an alarm lamp, and numeral 611 a surveillance monitor. The TV camera 601 is connected to the image input l/F 
602. the alarm lamp 610 is connected to the output l/F 607, and the monitor 611 is connected to the image output l/F 
608. The image input l/F 602. the image memory 603. the work memory 604. the CPU 605, the program memory 606 
the output l/F 607 and the image output l/F 608 are connected to the data bus 609. In Fig. 6. the TV camera 601 picks 
up an image in the image pickup view field including the area to be monitored. The TV camera 601 converts the image 
thus picked up into an image signal. This image signal is input to the image input l/F 602. The image input l/F 602 con- 
verts the input image signal into a format for processing in the object tracking system, and sends it to the image memory 
603 through the data bus 609. The image memory 603 stores the image data sent thereto. The CPU 605 analyzes the 
images stored in the image memory 603 through the work memory 604 in accordance with the program held in the pro- 
gram memory 606. As a result of this analysis, information is obtained as to whether an object has entered a predeter- 
mined monitoring area (for example, the neighborhood of a gate along a road included in the image pickup view field) 
in the .mage pickup view field of the TV camera. The CPU 605 turns on the alarm lamp 610 through the output l/F 607 
from the data bus 609 in accordance with the processing result, and displays an image of the processing result, for 
example, on the monitor 61 1 through the image output l/F 608. The output l/F 607 converts the signal from the CPU 
605 into a format usable by the alarm lamp 610. and sends it to the alarm lamp 610. The image output l/F 608 converts 
the signal from the CPU 605 into a format usable by the monitor 61 1 . and sends it to the alarm lamp 610 The monitor 

61 1 displays an image indicating the result of detecting an entering object The image memory 603, the CPU 605. the 
work memory 604 and the program memory 606 make up an input image processing unit. All the flowcharts below will 
be explained with reference to an example of the hardware configuration of the object tracking and monitoring system 
described above. 

[0038] Fig. 1 is a flowchart for explaining the process of updating the reference background image and detecting an 
entering object according to an embodiment of the invention. The process of steps 101 to 106 in the flowchart of Fig 1 
will be explained below with reference to Fig. 7 which has been used for explaining the prior art. 
[0039] At time point to. an input image 701 shown in Fig. 7 corresponding to 320 x 240 pixels is produced from a TV 
camera 601 (image input step 101). Then, the difference in intensity for each pixel between the input image 701 and 
the reference background image 702 stored in the image memory 603 is calculated by a subtracter 721 thereby to pro- 
duce a difference image 703 (difference processing step 102). The difference image 703 is processed with a threshold 
Specifically, the intensity value of a pixel not less than a preset threshold value is converted into "255" so that the par- 
ticular pixel is set as a portion where a detected object exists, while the intensity value less than the threshold value is 
converted into "0" si that the particular pixel is defined as a portion where no detected object exists, thereby producing 
a binanzed image 704 (binarization processing step 103). The preset threshold value is the one for determining the 
presence or absence of an entering object with respect to the difference value between the input image and the refer- 
ence background image and set at such a value that the entering object is not buried in a noise or the like as a result of 
binarization. This value is dependent on the object to be monitored and set experimentally. According to an example of 
the embodiment of the invention, the threshold value is set to 20. As an alternative, the threshold value may be varied 
in accordance with the difference image 703 obtained by the difference processing. 

[0040] Further, a mass of area 705 where the brightness value is "255" is extracted by the well-known labeling method 
and detected as an entering object (entering object detection processing step 104). In the case where no entering 
object is detected in the entering object detection processing step 104. the process jumps to the view field dividing step 
201 . In the case where there is an entering object detected, on the other hand, the process proceeds to the alarm/mon- 
itor indication step 106 (alarm/monitor branching step 105). In the alarm/monitor indication step 106 the alarm lamp 
610 k turned on or the result of the entering object detection process is indicated on the monitor 61 1 . The alarm/mon- 
itor indication step 106 is followed also by the view field dividing step 201. Means for transmitting an alarm as to the 
presence or absence of an entering object to the guardsman (or an assisting living creature, which may be the guards- 
man himself, in charge of transmitting information to the guardsman) may be any device using light electromagnetic 
wave, static electricity, sound, vibrations or pressure which is adapted to transmit an alarm from outside of the physical 
body of the guardsman through any of his sense organs such as aural, visual and tactile ones, or other means oivina 
rise to an excitement in the body of the guardsman. 

[0041] Now. the process of steps 201 to 205 in the flowchart of Fig. 1 will be explained with reference to Figs. 5. 7 and 
8. 
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[0042] In the view field dividing step 201. the view field is divided into a plurality of view field areas, and the process 
proceeds to the image change detection step 202. Specifically, the process of steps 202 to 205 is repeated for each 
divided view field area 

[0043] In the view field dividing step 201 , division of the view field is previously determined based on, for example, an 
5 average moving distance of an entering object, a moving direction thereof, for example, parallel to the moving direction 
(for example, traffic lanes when the entering object is a vehicle) or perpendicular thereto, a staying time of an entering 
object or the like. Other than these, by setting dividing border lines to border portions existing in the monitoring view 
field (for example, a median strip, a median line, a border line between roadway and sidewalk or the like when a moving 
object is a vehicle moving on a road), it becomes possible to make mismatching portions between those pixels of the 
w reference background image that are updated and those not updated harmless. Other than those dividing portions 
mentioned above, the view field may be divided at any portions that may possibly cause intensity mismatching such as 
wall, fence, hedge, river, waterway, curb, bridge, pier, handrail, railing, cliff, plumbing, window frame, counter in a lobby, 
partition, apparatuses such as ATM terminals, etc. 

[0044] The process from the image change detection step 202 to the divided view field end determination step 205 
is is executed for each of the plurality of divided view field areas. Specifically, the process of steps 202 to 205 is repeated 
for each divided view field area. First, in the image change detection step 202, a changed area existing in the input 
image is detected for each divided view field area independently. Fig. 5 is a diagram for explaining an example of the 
method of processing the image change detection step 202. In Fig. 5, numeral 1001 designates an input image at time 
point to-2, numeral 1002 an input image at time point to~1, numeral 1003 an input image at time point to, numeral 1004 
20 a binarized difference image obtained by determining the difference between the input image 1002 and the input image 
1003 and binarizing the difference, numeral 1005 a binarized difference image obtained by determining the difference 
between the input image 1003 and the input image 1002 and binarizing the difference, numeral 1006 a changed area 
image, numeral 1007 an entering object detection area of the input image 1001 at time point to-2, numeral 1008 an 
entering object detection area of the input image 1002 at time point to-1, numeral 1009 an entering object detection 
25 area of the input image 1003 at time point to. numeral 1010 a detection area of the binarized difference image 1004, 
numeral 101 1 a detection area of the binarized difference image 1005. numeral 1012 a changed area, numerals 1021, 
1022 difference binarizers. and numeral 1023 a logical product calculator. 

[0045] In Fig. 5. entering objects existing in the input image 1001 at time point to-2. the input image 1002 at time point 
to-1 and the input image 1003 at time point to are indicated as a model, and each entering object proceeds from right 
30 to left in the image. This image change detection method regards time point to as the present time and uses input 
images of three frames including the input image 1 001 at time pint to-2, the input image 1002 at time point to-1 and the 
input image 1003 at time point to stored in the image memory 603. 

[0046] In the image change detection step 202, the difference binarizer 1021 calculates the difference of the intensity 
or brightness value for each pixel between the input image 1001 at time point to-2 and the input image 1002 at time point 
35 to-1 . and binarizes the difference in such a manner that the intensity or brightness value of the pixels for which the dif- 
ference is not less than a predetermined threshold level (20. for example, in this embodiment) is set to "255", while the 
intensity value of the pixels less than the predetermined threshold level is set to "0". As a result the binarized difference 
image 1004 is produced. In this binarized difference image 1004. the entering object 1007 existing in the input image 

1001 at time point V 2 * s overlapped with the entering object 1008 existing in the input image 1002 at time point to-1, 
40 and the resulting object is detected as the area (object) 1 01 0. In similar fashion, the difference between the input image 

1 002 at time point V 1 and the input image 1003 at time point V> is determined by the difference binarizer 1022 and bina- 
rized with respect to the threshold level to produce the binarized difference image 1005. In this binarized difference 
image 1005, the entering object 1008 existing in the input image 1002 at time point to-1 is overlapped with the entering 
object 1009 existing in the input image 1003 at time point V). and the resulting object is detected as the area (object) 

45 1011. 

[0047] Then, the logical product calculator 1023 calculates the logical product of the binarized difference images 
1004. 1005 for each pixel thereby to produce the changed area image 1006. The entering object 1008 existing at time 
point to-1 is detected as a changed area (object) 1012 in the changed area image 1006. As described above, the 
changed area 1012 with the input image 1002 changed by the presence of the entering object 1008 is detected in the 

so image change detection step 202. 

[0048] In Fig. 5. a vehicle enters or moves, and this entering or moving vehicle is produced as the changed area 1012. 
[0049] The image change detection method described with reference to Fig. 5 is disclosed in H. Ohata et al. "A 
Human Detector Based on Flexible Pattern Matching of Silhouette Projection" MVA 94 IAPR Workshop on Machine 
Vision Applications Dec. 13-15. 1994. Kawasaki, the disclosure of which is hereby incorporated by reference. 

55 [0050] At the end of the image change detection step 202. the input image 1002 at time point to-1 is copied in the area 
for storing the input image 1001 at time point to-2 in the image memory 603. and the input image 1003 af time point to 
is copied in the area for storing the input image 1002 at time point in the image memory 603 thereby to replace the 
information in the storage area in preparation for the next process. After that, the process proceeds to the division 
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update process branching step 203. 

[0051 J As described above, the image change between time Doints at whirh »h Q i™. * 

obtained can be detected from these input images in the^gec^e ZSTSTJ? T ** 

the dh,isfon update process t^^S^S^SXi ^the^e^S ^ determina « on »* h 
theother hand, the process branch S relere^r^^^lcS 68 ^ * ^ " 

- «-ge at time point ,0-1. and the procel I^JTZTI ^ * ^ *° ^ 

reference background image update step 204 the update rate S £ JT? ^ . d f ern ***» s,e P 205. In the 
because the absence of the image change in the IE%L *L^£^ J° 3 M *™ in * e **» -» 
detection step 202 and the dhSon ^VXl^^^^T^^f^ T Change 
amount of the update processing from the time of an in™ * i.™? k . 1 * Update rate '"wives only a small 

« referencebacCnd'Lge. CSSSL^ 

process can be completed only with four frames from^he occurrence J ^ 1^™* J ^"P' 6 ' "P** 8 
reference background image. Thus the reference backgrou^Tcan be SiS^T 8 *.? " Updati " 9 * »* 
when the entering object detection process is executed anh! I Upda!ed wrth,n ,ess than °"e second even 

iment. the reference background imag^SufrSS^ nL^^ jf per second. Accorting to this embod- 

* time man in the prior art and *J^£^££^ 66 ^ * Shorter 
nance of the view fieW environment urtm£!Z*£ ***** ewBB "» 3 80606 where the ■»* 

where the process of the imageTange fdeteSon^ Sto ^rS^^omu^nT ™ ~ h the °~ 
ing step 205 has been ended for all the divided viewf ield ZT£ ^ ,ma9e d,v,s,on "P*** Process- 

input step 101 . and the series of P^sTS^^^^^^J^ process returns * •» image 
divided view field end determination step 205 Sh S2S2?S £ ""^ °° w after 1,16 

[0056] in the embodiment described above the view field hk/Hoh in*« x 

step 201. and the reference background image is ^^Z^Z^lT' ^ 
ence background image division update processing^ 2M SenlnThe Z^IZT m refer - 

in a portion of the view field, therefore the reference ba^omu^ tmlnl T * a " ,mage chan 9 e has ocam «i 

* other than the change area. Also, ft ScTo, m^of"^ "* — 
prxels updated and not updated which occurs only in the Zu^^^^^l^TP * 
reference background image updated by the dynamic area upSng TJ fJlTS f "" ln ^ 
•mage required for detecting entering objects can be updated * ?«ZtT" ! reference background 

s [0057] Another embodiment of the invention will be explained wiih rrf^ ^Tc , , ^' 

fieW is divide into a plurality of areas and the a-^5S22C^t ^ ^ 

area. Rg. 2 is a flowchart for explaining the Drocess of nivteti™ 2T ® executed for each divided view field 

entering object according to thfe^nW^^tt^l^ „ ^ back S round >™** and detecting an 
flowchart of Rg. 1 is ex<Led SSS^^X S T^bT T ^ * P 2 ° 1 ^ 

' entering object detection processing step 104 is replaced byl dSSf » JrL b ' nan2a,lon s,e P 103. Further, the 
an entering object in each divided view JMdll ^ XSl! WeW ,,eld ^ de,ection s,e P 301 for detecting 

field area alarm/monitor branch^ CtSSt ra " Chin9 ^ 105 * rep,aced ^ a vta! 
divided view field area, and the ZK£S£ES? 1 « S^LT^ " ^ *» "* 

-ndlcation step 303 for issuing or indicating an a.arm on a r^offoVeSS 

areas covered by the divided view field area detection sten -?ni ■ v,ew,ieW area - 17,6 ^"ded view field 

step 302 and the alarm/monitor ind^S^e dSv^il' tTjZ TJZ?^™^ 
he vew field covered by the entering object detection proces^ng^ ^ 20 \ «* dividin 9 

the alarm/monitor indication step 106. respectively. alarm/monitor branchmg step 105 and 
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[0058] As described above, according to this invention, the reference background image is updated for each divided 
view field area independently, and therefore the mismatch described above can be avoided in each divided view field 
area. Also, since the brightness mismatch occurs in the known boundary of divided view field areas in the reference 
background image, an image memory of small capacity can be used and further it can be easily determined from the 
5 location of a mismatch whether pixels that are detected are caused by the mismatch or really correspond to an entering 
object so that the mismatch poses no problem in object detection. In other words, the detection error (the error of the 
detected shape, the error in the number of detected objects, etc.) which otherwise might be caused by the intensity mis- 
match between the pixel for which the reference background image can be updated and the pixel for which the refer- 
ence background image cannot be updated can be prevented and an entering object can be accurately detected. 

10 [0059] Still another embodiment of the invention will be explained with reference to Figs. 3A, 3B. Fig. 3A shows an 
example of a lane image caught in the image pickip view field of the TV camera 601, and Fig. 3B shows an example 
of division of the view field. In this example, the view field is divided based on the average direction of movement of 
entering objects measured in advance in the view field dividing step 201 of the flowchart of Fig. 2, in which the objects 
to be detected by monitoring a road are automotive vehicles. Numeral 401 designates a view field, numeral 402 a view 

75 field area, numerals 403, 404 vehicles passing through the view field 401 , numerals 405, 406 arrows indicating the aver- 
age direction of movement, and numerals 407, 408, 409, 410 divided areas. 

[0060] In Fig. 3A, the average direction of movement of the vehicles 403, 404 passing through the view field 401 is 
as shown by arrows 405, 406, respectively. This average direction of movement can be measured in advance at the 
time of installing the image monitoring system. According to this invention, the view field is divided in parallel to the aver- 
se age direction of movement, as explained below with reference to Figs. 1 1 A to 1 1C. Numeral 1101 designates an exam- 
ple of the view field divided into a plurality of view field areas, in which paths 1101a, 1 1 01 b of movement of the object 
to be detected obtained when setting the monitoring view f ield are indicated in overlapped relation. The time taken by * 
an object entering the view field before leaving the view field along the path 1101a is divided into a predetermined 
number (four, in this example) of equal parts, and the position of the object at each time point is expressed as a1, a2, 
25 a3, a4, a5 (the coordinate of each position is expressed by (X a1 . X a1 ) for a1 , for example). Also, the vectors of each sec- 
tion are expressed as a21 , a32. a43, a54 (anm represents a vector connecting a position an and a position am). In sim- 
ilar fashion, the time taken by an object entering the view field before leaving the view field by plotting the path 1 101b 
is divided into a predetermined number of equal parts, and the position of the object at each time point is expressed as 
b1 , b2, b3, b4, b5 (the coordinate of each position is expressed as (X^, Y b1 ) for b1 , for example). The vector of each 
30 section is given as b12, b23, b34, b45 (bnm indicates a vector connecting a position bn and a position bm). Thus, the 
vector of each section indicates the average direction of movement. Also, the intermediate points between the positions 
a1 and b1 , the positions a2 and b2, the positions a3 and b3, the positions a4 and b4 and the positions a5 and b5 are 
expressed as d, c2, c3, c4, c5, respectively, (the coordinate of each position is expressed as (X^, Y C1 ) for c1, for 
example). In other words, X c1 =(X a1 +X b1 )/2, Y C1 =(Y a1 + Y b1 )/2(i:1 to 5). The line 1102c connecting the points 
35 ci thus obtained is assumed to be a line dividing the view field (1 102). Thus, the view field is divided as shown by 1 103. 
Also, even in the case where there are not less than three routes of movement of objects, the moving paths of the 
objects following adjacent routes are determined in the same manner as in the case of Figs. 11 A to 1 1 C. 
[0061 ] In the case of the view field 401 , for example, as shown by the view field area 402, the image pickup view field 
is divided into areas 407, 408, 409, 410 by lanes. Entering objects can be detected and the reference background 
40 image can be updated for each of the divided view field areas. Therefore, even when an entering object exists in one 
divided view field area (lane) and the reference background image of the divided view field area of the particular lane 
cannot be updated, the reference background image can be updated in other divided view field areas (lanes). Thus, 
even when an entering object is detected in a view field area, the reference background image required for the entering 
object detection process in other divided view field areas can be updated within a shorter time than in the prior art 
45 shown in Fig. 2. In this way, even on a scene where the illuminance of the view field area undergoes a change, an enter- 
ing object can be accurately detected. 

[0062] A further embodiment of the invention will be explained with reference to Figs. 4A, 4B. Fig. 4A shows an exam- 
ple of a lane image caught in the image pickup view field of the TV camera 601 , and Fig. 4B shows an example division 
of the view field). This embodiment represents an example in which the view field is divided in the view field dividing 
so step 201 of the flowchart of Fig. 2, based on the average distance coverage of an entering object measured in advance, 
and the object to be detected by monitoring a road is assumed to be a vehicle. Numeral 501 is a view field, numeral 502 
a view field area, numeral 503 a vehicle passing through the view field 501 , numeral 504 an arrow indicating the aver- 
age distance coverage, and numerals 505, 506, 507, 508 divided areas. 

[0063] In Fig. 4A, the moving path of the vehicle 503 passing through the view field 501 is indicated by arrow 504. 
55 This moving path can be measured in advance at the time of installing an image monitoring system. According to this 
invention, the view field is divided into equal parts by an average distance coverage based on object moving paths so 
that the time taken for the vehicle to pass through each divided area is constant This will be explained with reference 
to Figs. 12A, 12B and 12C. Numeral 1201 in Fig. 12A designates an example of the view field area to be divided, in 
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divided into a predetermined number of equal Darts and the notion m Lk.'11 \° 9 me v,ew ,,eld s 

.1. e2. e3 . e4. e5 (the coordinate of J*££i 2£lS2S *"? ft! * 38 

ment of each position represents the ■JS^TS^^^V^ ^ ' ** diSp,aCe - 

u" TT e f • T 0 " 8 0,3 and 63 SSiSTs S^SSSST £ 1? 

- L4. L5, respectively. In other words, the relation Li:y = (y „ - y H .)/ (x . - x i) x ix - * \i ? ' ' 

where each value U obtained is assumed as a line diking the view fi « -nf. I* ( *? 15 aSSUmed ' 
shown by 1203. Also when three of more routes of n»Z tS IT5*2ffi ! ^ !! * 88 
andthe divtfed areas can be determined in a ~ZT to °^^^t^T^ T* **** 
[0064] In the example of the view field 50 1 , as shown in the view field am* ^9 th Qi 
*<^^areas505.50^ 

•ng object ,s detected and me reference background image is updated for ea^Z^^^ T^T 
the case where an entering object exists in one lane, the entering object detecton ~ T 'f 6 " " 

divided view field areas other than the area where the errterina Silt «S ^TJTJL ** ewculed in *™ 

feren, colors on the screen of the monitor 61 1 . ££- Eto&T2J?Jl?J^ ^ * indiCated by * 
the screen. This is of course also the case with the SiS^B ~ °" 

[0065] Also, when monitoring other than a road, such as a harbor the view field can ho hw.^ ■ 
the time where an object stays in a particular area where the ^ITj^Z^t^^Z 
specrf .ed. such as at the entrance of a port, a wharf, a canal or straits. P TOtK>n ^ 

[0066] As described above, even when an enterirm nhi<ar* ic Ho^^ • _ - ' 
image required for the entering object deTec^ 

shorter time than in the orior art shown in Fin i -iw • Z -T Ms 030 be u P da,ed """im a 

where the luminance ^^^Z ^S^ *~* be aCCUra,ely detecled in « 

[0067] In other embodiments of the invention the view fiein ic Hh,.H~H k» 

ment and the average moving distance JSSS S SSencet XSSTS 4B 01 m ° Ve - 

iment of Fig. 3. the reference background image cannot be inSS in a St- . . ' ? & S P eo,,cal, K »» embod- 
Also, in the embodiment of Fig 4 inTe ^T^^nl^^^ ^ ***** ™ Bering object exists, 
reference background image ca^ u^Z ^^s^^T T °' °' "» road - *» 

and several areas, howev*. the ^^^Z^tZ^^l^^^T ^ 
than the lane or the segment where the entering object existe^re^ JlTtS f eW *** 0ther 
detected in a given view f few area, the refereni^Sima-^^ ^ l*™^™ 9 °* cl * 
can be updated in a shorter time than in the prior art in'other than The ^^S^^IT^ "T* 
ing object exists. In this way, an entering object can be accural ItZ^I here 6 partcular entef - 

the view field environment changes. accurately detected even m a scene where the illuminance of 

I0068J As described above, according to this invention, even in the case where an ont^™ „k.~. 

view field area the reference background image required for the emen^ f ^ ^ " deleCted ,n a 

divided view field area where the particular entering *jeS Z^^^^^^JT^ n °! her *" *• 

the reference background image by the convention!. Sd-u^nW FuX me Waning 6 ^ 2?" 

that can be updated and pixels in the divided view field ar^W t ^^!^f^^ m,STOteh between ** xeis 

conv entiona. dynamic area updating method. C « ^^^^2", 2,^^ ""^ «» 

where the illuminance of the view field environment undergoes a change accurate, y detected ^n in a scene 

in the prior art usins an imaoe mamory of a levw, rs^ritv b,hhT^ V» mag< ' w,l, " n " shorte ' lim = I"™ 

[0070] The method for updating the reference background imaae and tho m^nH w h^-^ _ . 
according to the invention described above can be executedksT^l^ ? detect,n9 entenn9 ob ' ecte 
computer readable medium. SOftware product a « a program realized on a 
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Claims 

1 . A method of updating a reference background image for use in detecting one or more objects entering an image 
pickup view field based on a difference between an input image and the reference background image of said input 

5 image, comprising the steps of: 

dividing said view field into a plurality of view field areas (201); and 

detecting a change in a video signal of each part of said input image corresponding to each of said divided view 
field areas independently (202); and 
w updating a part of said reference background image corresponding to that part of said input image which has 

no image signal change (204). 

2. A method according to claim 1 , wherein said change includes a movement of an entering object 

15 3. A method according to claim 1 , further comprising the step of determining whether an entering object exists or not 
independently for each of said divided view field areas (302), 

wherein the steps of detecting and updating are applied to each of those divided view field areas that are deter- 
mined to have no entering object. 

20 4. A method according to claim 1 , wherein said dividing step includes the step of dividing said image pickup view field 
by one or more boundary lines generally parallel to the direction of movement of said entering objects. 

5. A method according to claim 1 , wherein said divkfing step includes the step of dividing said image pickup view field 
within the average movement range of said entering object during each predetermined unit time. 

25 

6. A method according to claim 1 , wherein said dividing step includes the step of dividing said view field by one or 
more boundary lines generally parallel to the direction of movement of said entering object and further subdividing 
said divided view field in an average movement range of said entering object during each predetermined unit time. 

30 7. A method according to claim 1 , wherein said input image includes a lane, and said dividing step includes the step 
of dividing said image pickup view field by one or more said lane boundaries. 

8. A method according to claim 1 , further comprising the step of displaying the boundaries of said divided view field 
areas on a display screen. 

35 

9. A method according to claim 1 , further comprising the step of displaying said divided view field areas on a display 
screen in different colors. 

10. A method according to claim 1 , wherein said input image includes at least a lane and said dividing step includes 
40 the step of dividing said image pickup view field by an average movement range of vehicles during each predeter- 
mined unit time. 

A method according to claim 1, wherein said input image includes at least a lane and said dividing step includes 
the step of dividing said image pickup view field by one or more lane boundaries and the step of subdividing said 
divided image pickup view field by an average movement range of the vehicle during each predetermined unit time. 

A method according to claim 1 , further comprising the step (104) of determining whether an entering object exists 
in said image pickup view field or not, said dividing step (201) and said step (204) of updating the reference back- 
ground image portion being executed after executing said determination step. 

A method according to claim 1 , wherein said dividing step divides said image pickup view field into a plurality of 
view field areas based on at least one of the average direction of movement of said entering object and the distance 
coverage of said entering object during a predetermined unit time. 

A system for updating the reference background image for use in the detection of objects entering the image pickup 
view field based on the difference between an input image and the reference background image of said input 
image, comprising: 
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a dividing unit (201) for dividing said image pickup view field into a plurality of view field areas- and 

an updating unit (204) for updating the part of the reference background imaae con^nrii™ < n ^ 

P.ura.ity of the divided view field areas independent for each ! ° "* ° f ^ 

1 & tZ^l 0 ^^ t0 i! aim U - Wherei " ** dividin9 unit < 201 ) said image pickup view field into a oluralHv 
of v,ew held areas based on at least one of the average direction of movement of said eSTooS £i £ 5? 
tance covered by sad entering objects for each predetermined unit time. entenng objects and the dis- 

16. A method according to daim 14, wherein said updating unit includes: 

an image change detection unit (202) for detectina the channp of tho ; m ~.^ ^ ; „„ , , , 

siding to each of said d.tfedUwU ^Z^t^l^J^^^Z 

a background .mage updating unit (204) for updating the part of said reference backgrou^ 

mg to each of sad drwded view field areas where the image signal has not changed espo 

17. A computer readable medium having computer readable program code means Pmfw^ ^ ^ 

or more entering objects in the image pickup view field bas2 rthTd^C!r^ detectn 9 °" e 

erence background image of said i^i^ °" betW6en an "»»* ima 9 e and • «* 

code means (201) for dividing said image pickup view field into a plurality of view field areas- and 

o^rJ^ ^ ref — G ^">-f ^age cone^ng to eal of ^ plu . 

rally of the d.v.ded view f .eld areas .ndependentjy for each of said divided view field areas. 

1a A method according to claim 17. wherein said updating code means (204) includes: 

code means (202) for detecting the change of said input image within each of saw m, , ra i«„ «* * ■ 
field areas independently for each of said derided view field aW and ^ ^ V,W 

SJ?^ f ° w 4> ** 09 ^ 01 53111 re,erence ground image corresponding to each of said 

1 9. A method of detecting entering objects, comprising the steps of: 

detecting one or more objects entering an image pickup view field based on thp riiff*™,™ k«k 

image and a reference background image of said input ^(104) 

drnding said image pickup view field into a plurality of view field areas (201)- and 

2a A method according to claim 1 9. wherein said step of updating sakt reference background image includes the steps 
dividing said image pickup view field into a plurality of view field areas (201) 

tSSSlSS^S ° f If ' ma9e *** ° f * 6 inpUt ima9e t**™ responding to each of said divided view 
f -eld areas (202) .ndependentjy for each of said divided view field areas- IZt ^ *" 



updating the portion of said reference background image corresponding to each of said divided view field 
correspond**, to sa.d .nput image portion in which the image signal has not change (2oT 

21. A method according to claim 19. wherein said change of said image signa. is toe movement of an entering object. 

^ ft^!! h0d aCCOrdin9 *° <**" 19 - whe '* n sa« dividing step includes the step of dividing said imaoe oi 
few by one or more boundary lines generally parallel to the direction of movement of sal. XSSSE 



pickup view 
entering object. 



23. A method according to claim 19. wherein said dividina sten indiirioc th* ~* ^ 

•*> * - a^age movant ^ * sa M *J^tt£2£2ES£ **" *" 

24. A method acceding to claim ,9. *ba,ain said dividing step includes the ste p of dividing said view Md by ona o, 
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more boundary lines generally parallel to the direction of movement of said entering object and subdividing said 
divided view field by an average movement range of said entering object during each predetermined unit time. 

25. A method according to claim 19, wherein said input image includes at least a lane and said dividing step includes 
5 the step of dividing said image pickup view field by one or more lane boundaries. 

26. A method according to claim 19. wherein said input image includes a lane, and said dividing step includes the step 
of dividing said image pickup view field by an average movement range of a vehicle during a predetermined unit 
time. 

w 

27. A method according to claim 19. wherein said input image includes a lane and said dividing step includes the step 
of dividing said image pickup view field by one or more lane boundaries and subdividing said divided image pickup 
view field by an average movement range of the vehicle during a predetermined unit time. 

75 28. A method according to claim 19, wherein said dividing step divides said image pickup view field into a plurality of 
view field areas based on at least one of the average direction of movement of said entering object and the distance 
covered by said entering object during a predetermined unit time. 

29. A method of detecting entering objects, comprising the steps of: 

20 

dividing said image pickup view field into a plurality of view field areas (201); and 

detecting one or more objects entering an image pickup view field based on the difference between an input 
image and the reference background image of said input image (104) and updating said reference background 
image for detecting the next entering object (204). 

25 

30. A method according to claim 29, wherein said step of updating said reference background image includes the steps 
of: 

detecting the change of the image signal of the input signal corresponding to each of said divided view field 
30 areas (202); and 

updating said reference background image corresponding to said divided view field area in the absence of the 
change of the image signal (204). 

31. A method according to claim 29, wherein said dividing step includes the step of dividing said image pickup view 
35 field by the average movement range of said entering object during each predetermined unit time. 

32. A method according to claim 29, wherein said dividing step includes the steps of dividing said view field by one or 
more boundary lines generally parallel to the direction of movement of said entering object and subdividing said 
divided view field areas by the average movement range of said entering object during each predetermined unit 

ao time. 

33. A method according to claim 29, wherein said input image includes at least a lane and said dividing step includes 
the step of dividing said image pickup view field by one or more lane boundaries. 

45 34. A method according to claim 29, wherein said input image includes at least a lane, and said dividing step includes 
the step of dividing said image pickup view field by the average movement range of a vehicle during a predeter- 
mined unit time. 

35. A method according to claim 29, wherein said input image includes at least a lane, and said dividing step includes 
so the step of dividing said image pickup view field by one or more lane boundaries and subdividing said divided 

image pickup view fields by the average movement range of a vehicle during each predetermined unit time. 

36. A system for detecting entering objects, comprising: 

55 an image input unit (601 , 602); and 

a processing unit including an image memory (603) for storing the input image from said image input unit, a 
program memory (606) for storing the program for operation of said entering object detecting unit and a central 
processing unit (605) for activating said entering object detecting unit in accordance with said program for 
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processing said input image; 
said processing unit including: 

K^ ri " 9 f i6C !. deteCti ? Ulit (102 t0 104: 102> 103 ' 301) *" deter**** the difference for each pixel 
between sad .nput .mage from said image input unit and the reference background image not including an 

^asSe^^ 

a dividing unit (201) for dividing the image pickup view field of said image input unit into a plurality of view field 
areas, 

an image change detecting unit (202) for detecting the change of the image in each of said divided view field 
areas, and 

a reference background image updating unit (204) for updating each portion of said reference backoround 

sssssass - ,feu area ™°*™*> » — « - 

g^nd n irnage enterin9 ***** mit detects m based on said updated reference back- 

37. A computer readable medium having computer readable program code means embodied therein for detectina 
entering objects in an image pickup view field, comprising: aetecting 

code means (102 to 103) for detecting the intensity difference for each pixel between an input image and the 
present reference background image of said input image- 

X^ZvZl; and' 61606 " 9 ~" " * *» «*" °" sakf 

code means (204) for updating said present reference background image tor detecting the intensity difference 
for each p.xel between a newly input image and said reference background imager^ * fference 

wherein said code means for updating said reference background image includes-' 
code means (201) for dividing said image pickup view field into a plurality of view field areas- 
code means (202) for detecting the change of the image signal of the input image portion conespondina to 
each of sa,d pluralrty of said divided view field areas independently for each of said ^^SSStK 

codemeans (204) for updating the portion of said reference background image corresponding to each of said 
tZf™ ^ COrreSp0ndin9 to *• P° rton °< «*» in P* image of which saTimage ^Thas^ 

38. A computer readable medium having computer readable program code means embodied therein for detectina 
entering objects in an image pickup view field, comprising: mere,n detect,n 9 

co^means (201) for dividing the image pickup view field of an image pickup device into a plurality of view field 

code means (301) for detecting one or more objects entering said divided view field areas based on the inten- 
sity difference for each pixel between the image input from said image pickup device and me SeTence baS- 
ground image independently for each of said divided view fields- and reierence Dack 

code means (204) for updating said reference background image corresponding to each of said divided view 
fields for detecting the intensity difference between a newly input image and said reference bS™5tZ 
independently for each of said divided view field areas. uacKgrouna image 

39 ' tocTut? aCCOrdin9 10 Claim 38> Wh6rein COde meanS *» ***** ^ re ^e background image 

sS e dS s r ito rr chan9e of the ima9 ° * *• ^ ~« *> — of 
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